The consumer financial products industry consists of tens of thousands of financial institutions offering a wide variety of individual financial products to consumers, almost all of which advertise and offer their services over the world wide web (hereinafter the “Web”). For example, currently, there can be found over 15,000 FDIC and NCUA-insured banks and credit unions offering financial services in the US, almost all of which offer deposit products to consumers and businesses. These institutions offer on average over a dozen financial products, each having varying tiers with separate interest rates. The result is well over a quarter million different interest bearing products in the US alone. However, tracking the interest rates offered by these institutions for each of their products and trying to get a sense of what is a reasonable or competitive rate among these institutions is a challenge. Further, compiling trend data for these interest rates is even more challenging, requiring hundreds of man hours to input and correlate such rate data. This causes delays in compiling trend data and seriously diminishes the usefulness of any such trend analysis.
There are some services on the Internet that track interest data and publish averages and trends on their websites. These “rate data aggregators” as they are known acquire their rate data primarily through direct interactions with financial institutions, and often are the result of long-standing relationships with the financial institutions offering the products that are tracked. However, inefficiencies and challenges with current rate product data acquisition methods lead to costly, incomplete, and out-of-date rate data. The rate data is typically gathered via a combination of scattered, and mostly manual processes, such as for example: call centers staffed by teams of callers to contact and record rates offered by selected institutions; email requests that are transmitted with a cataloging of replies; collection of rate data using voluntary FTP connections; accessing secure databases; and the recordation of data received in broadcast faxes from institutions. These manual collection strategies result in numerous human born errors, and are typically not relevant to current rates because of the delay between the collection of the data and the inputting and correlation of that rate data. Also, these collection strategies are relegated to a sample size attainable by human collection efforts, and do not allow the assimilation of a high percentage of available rates. Finally, much of the data is dependent upon an interactive and cooperative relationship between the financial institution offering a product and the collection service wishing to obtain the rate data. Such relationships can deteriorate rapidly, and require resources to maintain. Moreover, human collection efforts are relatively expensive, thereby requiring a higher return on a data collection investment which can jeopardizing business continuity.
Therefore, what is needed is a system that monitors and parses financial institution websites for financial product rate data, especially focusing on products with frequently changing rate data, across a plurality of website technologies and display formats, and then publishes that data on the Web for easy consumer access. The system would also create a very large repository of structured rate data from which current rate data trends may be ascertained in real-time to licensed subscribers.